Water upstream are cool and appear to be safe to drink.
While this is true for natural springs that dot the countryside, anyone venturing into the backcountry should bring a backpacker's water filter and/or be prepared to boil water.
Wamena city water is NOT safe to drink unless boiled and filtered. Hotels and restaurants typically provide boiled and filtered water, and offer bottled water as a reasonable price. There are a number of local purification facilities that produce bottled water, generally available in 1/2 liter to 20 liter sizes.
Note that alcohol in any form is illegal and simply is NOT available in the Baliem Valley.